Relationships Between Support Vector Classifiers and Generalized Linear Discriminant Analysis on Support Vectors
نویسندگان
چکیده
The linear discriminant analysis based on the generalized singular value decomposition (LDA/GSVD) has recently been introduced to circumvents the nonsingularity restriction that occur in the classical LDA so that a dimension reducing transformation can be effectively obtained for undersampled problems. In this paper, relationships between support vector machines (SVMs) and the generalized linear discriminant analysis applied to the support vectors are studied. Based on the GSVD, the weight vector of the hard margin SVM is proved to be equivalent to the dimension reducing transformation vector generated by LDA/GSVD applied to the support vectors of the binary class. It has also been shown that dimension reducing transformation vector and the weight vector of soft margin SVMs are related when a subset of support vectors are considered. These results can be generalized when kernelized SVMs and the kernelized KDA/GSVD are considered. Illustrating the relationship, it is shown that a classification problem can be interpreted as a data reduction problem. Index Terms Machine Learning, Support Vector Machines, Generalized Linear Discriminant Analysis (LDA), Generalized Singular Value Decomposition February 15, 2004 DRAFT
منابع مشابه
A prediction distribution of atmospheric pollutants using support vector machines, discriminant analysis and mapping tools (Case study: Tunisia)
Monitoring and controlling air quality parameters form an important subject of atmospheric and environmental research today due to the health impacts caused by the different pollutants present in the urban areas. The support vector machine (SVM), as a supervised learning analysis method, is considered an effective statistical tool for the prediction and analysis of air quality. The work present...
متن کاملA prediction distribution of atmospheric pollutants using support vector machines, discriminant analysis and mapping tools (Case study: Tunisia)
Monitoring and controlling air quality parameters form an important subject of atmospheric and environmental research today due to the health impacts caused by the different pollutants present in the urban areas. The support vector machine (SVM), as a supervised learning analysis method, is considered an effective statistical tool for the prediction and analysis of air quality. The work present...
متن کاملTraining set approximation for kernel methods
We propose a technique for a training set approximation and its usage in kernel methods. The approach aims to represent data in a low dimensional space with possibly minimal representation error which is similar to the Principal Component Analysis (PCA). In contrast to the PCA, the basis vectors of the low dimensional space used for data representation are properly selected vectors from the tra...
متن کاملGeneralized Maximal Margin Discriminant Analysis for Speech Emotion Recognition
A novel speech emotion recognition method based on the generalized maximummargin discriminant analysis (GMMDA) method is proposed in this paper. GMMDA is a multi-class extension of our proposed two-class dimensionality reduction method based on maximum margin discriminant analysis (MMDA), which utilizes the normal direction of optimal hyperplane of linear support vector machine (SVM) as the pro...
متن کاملGreedy Algorithm for a Training Set Reduction in the Kernel Methods
We propose a technique for a training set approximation and its usage in kernel methods. The approach aims to represent data in a low dimensional space with possibly minimal representation error which is similar to the Principal Component Analysis (PCA). In contrast to the PCA, the basis vectors of the low dimensional space used for data representation are properly selected vectors from the tra...
متن کامل